Add method coverage support by tycooon · Pull Request #987 · simplecov-ruby/simplecov

tycooon · 2021-04-23T17:58:17Z

See #782.
Also addresses #801 and #916.
Support for HTML formatter can be found here: simplecov-ruby/simplecov-html#110.

Some notes:

I had to rework how coverage is parsed/unparsed (see ResultSerialization module).
Because of that, I had to revert some of ResultMerger optimizations :( Now it's operating Result objects, however it still parses and merges files one be one.
Coverage is now always stored in memory in symbolized form (lines: [] instead of "lines" => []), just the same way you get it when calling Ruby's Coverage.result. Because of that, a lot of specs were updated.
Sometimes method owner includes memory address (e.g. when method is called on anonymous class). This makes merging results from different CI jobs problematic, so we replace memory addresses with zero address 0x0000000000000000.
adapt_pre_simplecov_0_18_result logic was moved to ResultSerialization.deserialize. Also old-style branch info is supported (eval is still used for that).

All this stuff has been battle-tested on a couple of big projects, some of which merge results of multiple CI jobs.

…-support

tycooon · 2021-04-24T11:39:38Z

@PragTob hope you will find some time to review the changes.

pboling · 2023-03-29T21:03:26Z

@PragTob ping!

0exp · 2023-12-20T08:14:02Z

@PragTob ping!

…-support

gondalez · 2024-01-04T05:32:38Z

spec/fixtures/methods.rb

Thanks for your work on this PR, it would make a welcome inclusion 🙏

I am curious - would method coverage improve coverage support for ruby 3's endless methods?

I have found that endless methods currently show as covered if they are never executed by tests.
Even if you have branching coverage enabled they report as fully covered.

Example class, spec and coverage report follow.
Note that uncovered_endless_method shows as covered despite not being executed by the test.
Whereas endless_branching_method shows branching coverage failing since it was executed.

I would have expected uncovered_endless_method to show as partially covered.

class CoveragePoc # this method is fully covered def self.simple_method :simple end # the :b branch for these methods is not covered def self.endless_branching_method(arg) = arg ? :a : :b def self.branching_method(arg) arg ? :a : :b end # these methods are not covered at all def self.uncovered_endless_method = :uncovered_endless def self.uncovered_method :uncovered end end

RSpec.describe CoveragePoc do it 'works' do expect(described_class.simple_method).to eq(:simple) expect(described_class.branching_method(true)).to eq(:a) expect(described_class.endless_branching_method(true)).to eq(:a) end end

tycooon · 2025-11-03T08:22:24Z

Hey @amatsuda, maybe you could take a look? I know there are conflicts, but I don't want to fix them in case there are no plans to merge this PR anytime soon.

amatsuda · 2025-11-03T10:14:36Z

@tycooon Thank you for working on this! Yes, I'm very much interested in this feature, and willing to take a look once the patch is cleanly rebased against the main branch :)

…-support

tycooon · 2025-11-04T10:22:03Z

Hey @amatsuda, I just updated the branch 👍

sferik · 2026-03-27T23:34:18Z

lib/simplecov/result_serialization.rb

+      end
+
+      def adapt_old_style_branch_info(value)
+        eval(value.to_s) # rubocop:disable Security/Eval


I know this eval was in the original code, but I’d love to remove it.

sferik

@tycooon thanks for all your work on this PR! It's a great feature idea. I reviewed the code and left some minor feedback.

Reviewing this inspired me to take a crack at an alternative implementation that builds on the same ideas but makes a few different tradeoffs in add-method-coverage-support.

Here are main differences between my approach and yours:

Keeps string keys ("lines", "branches", "methods") instead of converting to symbols. This avoids the breaking change for downstream consumers like rubycritic that access coverage_data directly, and avoids touching nearly every spec file.
ResultMerger is completely untouched. The memory optimization of deferring Result object creation until the final merge step is preserved. Method data flows through the existing combine pipeline naturally since FilesCombiner/MethodsCombiner operate on raw hashes just like LinesCombiner/BranchesCombiner do.
Replaces eval with a safe StringScanner-based parser in restore_ruby_data_structure. This addresses #801 without requiring a separate ResultSerialization class. The same parser handles both branch and method data keys after JSON round-tripping.
Memory address normalization happens in ResultAdapter (early, before the JSON round-trip in Result.new) rather than during serialization.

Everything else is the same in spirit: the SourceFile::Method class, MethodsCombiner, configuration via enable_coverage :method, FileList aggregation, etc.

What do you think?

sferik · 2026-03-27T23:38:44Z

lib/simplecov/source_file.rb

+
+    def total_methods
+      @total_methods ||= covered_methods + missed_methods
+    end


The total_methods method defined in FileList returns a count, as the name suggests, but this one returns an array. Please change the name of this one (or change its behavior to be consistent with the other).

sferik · 2026-03-27T23:41:37Z

spec/source_file_spec.rb

+    end
+
+    describe "method coverage" do
+      it "has total methods count 0" do


Suggested change

it "has total methods count 0" do

it "has total methods count 3" do

sferik · 2026-03-27T23:41:55Z

spec/source_file_spec.rb

+        expect(subject.total_methods.size).to eq(3)
+      end
+
+      it "has covered methods count 0" do


Suggested change

it "has covered methods count 0" do

it "has covered methods count 2" do

sferik · 2026-03-27T23:42:09Z

spec/source_file_spec.rb

+        expect(subject.covered_methods.size).to eq(2)
+      end
+
+      it "has missed methods count 0" do


Suggested change

it "has missed methods count 0" do

it "has missed methods count 1" do

sferik · 2026-03-27T23:44:59Z

lib/simplecov/source_file/method.rb

+      end
+
+      def to_s
+        "#{klass}##{method}"


Ruby’s Coverage.result uses different prefixes for instance methods vs class methods (e.g., #<Class:Foo> for singleton methods). It should probably use using . instead of # for class methods.

sferik · 2026-03-27T23:49:56Z

lib/simplecov/source_file.rb

    end

+    def lines_data
+      coverage_data.fetch(:lines)


Is it possible for the :lines key to be missing (e.g. if the user just requests method coverage data)? If so, this will crash.

sferik · 2026-03-27T23:53:57Z

Gemfile.lock

+  arm64-darwin
+  x86_64-darwin
+  x86_64-linux-gnu
+  x86_64-linux-musl


This shouldn’t be part of this PR.

tycooon · 2026-03-28T06:52:24Z

Hey @sferik, I like your approach more than mine. I am closing this PR for now 👍

Also don't forget to add some support for HTML formatter. Example of what I did is here simplecov-ruby/simplecov-html#110.

And btw since you added your own parser, maybe we can address #916 as I did in the same branch?

tycooon added 21 commits August 22, 2020 13:08

add some todos

693af60

add method coverage criterion

2521b5c

add some temp stuff

d68a977

add basic method coverage support

1f40180

fix serialization

d609d16

some fixes and fix all specs

f00fdf2

add methods combiner

31b95e7

update ci config

fa3902a

Merge remote-tracking branch 'upstream/main' into add-method-coverage…

b929673

…-support

add some backwards compability to result serialization

a78b5ed

add some methods for html report

b16b6e5

Merge remote-tracking branch 'upstream/main' into add-method-coverage…

3ae4b18

…-support

revert simplecov-html version

66e6e0b

fix rubocop issues

d79cb80

fix flacky specs

4904192

Merge remote-tracking branch 'upstream/main' into add-method-coverage…

03d1180

…-support

minor refactor

3eff8b7

Merge remote-tracking branch 'upstream/main' into add-method-coverage…

efb3e20

…-support

fix results collation

9fafd3c

update test app gemfile

6420688

add specs

ddccb3c

tycooon mentioned this pull request Apr 23, 2021

Add method coverage support simplecov-ruby/simplecov-html#110

Open

optimize result merger

15c03b2

tycooon force-pushed the add-method-coverage-support branch 2 times, most recently from 9851ba0 to 5987457 Compare April 24, 2021 11:27

add specs for method cov merging

9a8b324

tycooon force-pushed the add-method-coverage-support branch from 5987457 to 9a8b324 Compare April 24, 2021 13:06

tycooon added 2 commits April 30, 2021 18:58

add feature specs for method coverage

dc754e4

fix indentation in features

b5c29b4

tycooon added 3 commits December 25, 2023 19:02

Merge remote-tracking branch 'upstream/main' into add-method-coverage…

7dca89d

…-support

fix rubocop issue

c13f0b1

revert bundler version

037a493

tycooon force-pushed the add-method-coverage-support branch from 2ce910b to 54684e8 Compare December 25, 2023 16:13

update simplecov-html

8ede952

tycooon force-pushed the add-method-coverage-support branch from 54684e8 to 8ede952 Compare December 25, 2023 16:28

sferik force-pushed the main branch 9 times, most recently from 33cbb8e to 574b45c Compare December 26, 2023 16:15

gondalez reviewed Jan 4, 2024

View reviewed changes

tycooon added 2 commits November 4, 2025 13:14

update branch

2811cc1

Merge remote-tracking branch 'upstream/main' into add-method-coverage…

c59387a

…-support

tycooon force-pushed the add-method-coverage-support branch from c2e0130 to c59387a Compare November 4, 2025 10:18

use gsub when replacing memory addresses

c8a8bc7

sferik reviewed Mar 27, 2026

View reviewed changes

sferik requested changes Mar 28, 2026

View reviewed changes

sferik force-pushed the main branch from 5561eb3 to 383c9ca Compare March 28, 2026 06:45

tycooon closed this Mar 28, 2026

	it "has total methods count 0" do
	it "has total methods count 3" do

	it "has covered methods count 0" do
	it "has covered methods count 2" do

	it "has missed methods count 0" do
	it "has missed methods count 1" do

Conversation

tycooon commented Apr 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tycooon commented Apr 24, 2021

Uh oh!

pboling commented Mar 29, 2023

Uh oh!

0exp commented Dec 20, 2023

Uh oh!

gondalez Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tycooon commented Nov 3, 2025

Uh oh!

amatsuda commented Nov 3, 2025

Uh oh!

tycooon commented Nov 4, 2025

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

sferik left a comment

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

sferik Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

tycooon commented Mar 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tycooon commented Apr 23, 2021 •

edited

Loading

gondalez Jan 4, 2024 •

edited

Loading

sferik Mar 27, 2026 •

edited

Loading